[CI] Upgrade CI #14635

yongwww · 2023-04-16T03:54:46Z

Component	Current Version	Upgraded Version
Docker of ci_cpu, gpu, arm, etc.	20.04	22.04
Docker of ci_i386	18.04	20.04 (No 22.04 so far)
CMake	3.18	3.20.0
LLVM	-	13, 14, 15, 16
ROCM	4.3	5.3 (4.3 has unmet packages in upgraded container)
CUDA	11.7	11.8
Python	3.7	3.8
Pylint	2.4.4	2.17.2
Torch	-	2.0
torchvision	-	0.15.1
libtorch	-	2.0
Golang	1.12	1.18
emsdk	2.0.7	3.1.30
Verilator	4.104	5.002
Numpy	1.21	1.23
DGL	0.7.2	1.0.0
clang-format	10	15

tvm-bot · 2023-04-16T03:54:50Z

Thanks for contributing to TVM! Please refer to the contributing guidelines https://tvm.apache.org/docs/contribute/ for useful information and tips. Please request code reviews from Reviewers by @-ing them in a comment.

cc @Mousius, @areusch, @driazati, @gigiblender, @leandron _{See #10317 for details}

_{Generated by tvm-bot}

tqchen · 2023-04-16T13:39:33Z

cc @masahi

docker/Dockerfile.ci_lint

leandron

Hi @yongwww, thanks for the PR. I see this is marked as work in progress.

Can I just ask the we validate the images in a full CI round before merging this PR, that not only they build the images correctly, but also that tests pass with the resultant images, as you're doing with #14651. These are a lot of changes for one PR, but I appreciate that moving versions is hard.

I'm marking this as "request changes" so that we don't accidentally merge this based on the local CI results.

yongwww · 2023-04-18T14:22:31Z

Thanks, @leandron ! I agree that this pull request should be held off until #14651 turns green. I've marked it as a work-in-progress and will revisit it

Aleksei-grovety · 2023-04-24T15:13:25Z

Hi @yongwww, thanks for the PR. I saw you tried tensorflow==2.12.0 but it was rolled back. Have you considered tensorflow==2.10? For example, Vela 3.7.0 (recently updated) supports TensorFlow 2.10.

yongwww · 2023-04-24T18:07:21Z

@Aleksei-grovety great observation! I rolled back the TensorFlow upgrade because the linking process failed for the TFLite runtime during the TVM build (lots of errors like "multiple definition ...", and "undefined reference to..."). This issues need specific update in the tflite runtime build, which I haven't had time to address. In fact, I think we should consider using TFLite runtime Python package (https://pypi.org/project/tflite-runtime/) provided by Google.

masahi · 2023-04-24T23:49:58Z

@yongwww Feel free to drop PT 2.0 update from this PR. I'm aware that some PT frontend tests are broken with 2.0. I can continue my PR to fix them, once we get Python 3.10 from here.

yongwww · 2023-04-25T00:10:08Z

@masahi I have fixed some of the failed pt tests in #14651, and skip those tests if there is new unsupported op. We will need PT 2.0 in unity branch, it would be good to have it in this upgrade

yongwww · 2023-04-27T05:03:44Z

The PR is ready for review, pls take a look. cc: @masahi @driazati @leandron @tqchen

driazati

Great work! Thanks @yongwww, the changes LGTM if the build from #14651 passes

yongwww · 2023-04-28T14:35:00Z

CI #14651 is green

tqchen · 2023-04-28T15:18:50Z

Thank you @yongwww for heroic effort!

psrivas2 · 2023-05-02T20:10:11Z

cc @leandron can you take another look at this PR? I think this is ready to be merged.

tqchen · 2023-05-03T00:28:48Z

ping @leandron

yongwww · 2023-05-03T03:10:35Z

@tvm-bot rerun

cautiously dismissing since the comments have been addressed and this PR has been in flight for a while, we can address any concerns in follow ups

leandron

I'm happy to merge this once #14651 CI is green. At the moment it is running, as the GitHub actions needed to be approved, which I just did.

I marked a few action I feel we need to take as follow-up PRs, can you just open those as issues so that we can pick them up asap. There no need to block this PR because the actual changes, but can you open the issues, please?

leandron · 2023-05-03T08:17:17Z

docker/install/ubuntu_install_ethosn_driver_stack.sh

@@ -54,4 +48,4 @@ cd "$tmpdir"
 git clone --branch "$repo_revision" "$repo_url" "$repo_dir"

 cd "$repo_dir"/driver
-scons install_prefix="$install_path" install
+scons werror=False install_prefix="$install_path" install


Do you mind opening an issue to report what is the error that makes this script now need werror=False?

I understand it will have some output, but can you add that output in the issue, so that somebody can pick that up to investigate?

I got error like "error: loop variable 'pair' creates a copy from type 'const std::pair<const unsigned int, std::pair<long unsigned int, long unsigned int> >' [-Werror=range-loop-construct] for (const auto pair : passAgentIdRanges)", it should be a warning, so I added werror=False in scons install for that, I think the ideal solution is to modify the code in https://github.com/Arm-software/ethos-n-driver-stack to fix this warning. More details about the error info. please refer to the log https://ci.tlcpack.ai/blue/rest/organizations/jenkins/pipelines/tvm-docker/branches/PR-14635/runs/5/nodes/73/steps/129/log/?start=0 Probably we can create an issue in the arm repo?

docker/Dockerfile.ci_arm

tqchen · 2023-05-03T20:22:18Z

Going to merge this first in 24 hours so we can proceed with the updates on the other side

tqchen · 2023-05-04T12:43:20Z

Thank you @yongwww for heroic effort

yongwww force-pushed the ci_upgrade branch from 9be797d to 37d92c1 Compare April 16, 2023 13:09

driazati reviewed Apr 18, 2023

View reviewed changes

docker/Dockerfile.ci_lint Show resolved Hide resolved

yongwww mentioned this pull request Apr 18, 2023

[CI] Modify test cases to accommodate the CI upgrades #14651

Merged

leandron previously requested changes Apr 18, 2023

View reviewed changes

yongwww force-pushed the ci_upgrade branch from 246f1ab to 984ef08 Compare April 22, 2023 00:52

yongwww changed the title ~~[WIP][CI] Upgrade CI~~ [CI] Upgrade CI Apr 24, 2023

yongwww added 14 commits April 25, 2023 16:00

[CI] Upgrade CI

16923cc

Update for GPU image

d88c11d

Fix docker build errors

bf2461d

Update Rocm

3f7adb0

Update for all dockerfiles

c1d5975

Fix ci_lint timeout

31135a3

Update ethosn_driver_stack and ignore werror in scons

1832d61

Update mxnet and onnx version

47e4c12

Update tflite version

0447b75

upgrade emsdk

34161bd

Delete unused scripts

8d1bf5e

update llvm build

9fc53a3

update numpy

f91ad2c

Upgrade verilator

22f634a

yongwww force-pushed the ci_upgrade branch from 58899c2 to cab75ba Compare April 25, 2023 23:01

Upgrade numpy

abdb99f

yongwww force-pushed the ci_upgrade branch from cab75ba to abdb99f Compare April 26, 2023 00:15

Upgrade DGL

16c0dce

driazati approved these changes Apr 27, 2023

View reviewed changes

tqchen approved these changes Apr 28, 2023

View reviewed changes

leandron reviewed May 3, 2023

View reviewed changes

tqchen merged commit 4d37a0a into apache:main May 4, 2023

yongwww deleted the ci_upgrade branch May 5, 2023 03:00

ysh329 mentioned this pull request Jul 12, 2023

[Release] v0.13.0 Release Candidate Notes #15295

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[CI] Upgrade CI #14635

[CI] Upgrade CI #14635

yongwww commented Apr 16, 2023 •

edited

Loading

tvm-bot commented Apr 16, 2023

tqchen commented Apr 16, 2023

leandron left a comment

yongwww commented Apr 18, 2023

Aleksei-grovety commented Apr 24, 2023

yongwww commented Apr 24, 2023 •

edited

Loading

masahi commented Apr 24, 2023

yongwww commented Apr 25, 2023 •

edited

Loading

yongwww commented Apr 27, 2023

driazati left a comment

yongwww commented Apr 28, 2023

tqchen commented Apr 28, 2023

psrivas2 commented May 2, 2023 •

edited

Loading

tqchen commented May 3, 2023

yongwww commented May 3, 2023

leandron left a comment

leandron May 3, 2023

yongwww May 3, 2023 •

edited

Loading

tqchen commented May 3, 2023

tqchen commented May 4, 2023

[CI] Upgrade CI #14635

[CI] Upgrade CI #14635

Conversation

yongwww commented Apr 16, 2023 • edited Loading

tvm-bot commented Apr 16, 2023

tqchen commented Apr 16, 2023

leandron left a comment

Choose a reason for hiding this comment

yongwww commented Apr 18, 2023

Aleksei-grovety commented Apr 24, 2023

yongwww commented Apr 24, 2023 • edited Loading

masahi commented Apr 24, 2023

yongwww commented Apr 25, 2023 • edited Loading

yongwww commented Apr 27, 2023

driazati left a comment

Choose a reason for hiding this comment

yongwww commented Apr 28, 2023

tqchen commented Apr 28, 2023

psrivas2 commented May 2, 2023 • edited Loading

tqchen commented May 3, 2023

yongwww commented May 3, 2023

leandron left a comment

Choose a reason for hiding this comment

leandron May 3, 2023

Choose a reason for hiding this comment

yongwww May 3, 2023 • edited Loading

Choose a reason for hiding this comment

tqchen commented May 3, 2023

tqchen commented May 4, 2023

yongwww commented Apr 16, 2023 •

edited

Loading

yongwww commented Apr 24, 2023 •

edited

Loading

yongwww commented Apr 25, 2023 •

edited

Loading

psrivas2 commented May 2, 2023 •

edited

Loading

yongwww May 3, 2023 •

edited

Loading